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ABSTRACT 

A classroom observation rating scale, based upon a 

recent analysis of the literature and conceptually verified by open 
education advocates, effectively differentiated British and American 
open classrooms from American traditional classrooms. The influence 
of socio-economic settings was also demonstrated. For the ree 



were found in 

Rater- reliability 



comparison groups, more features of open education 
higher socio-economic settings than in lower ones, 
for the classroom observation measure was high. The classroom < 
observation rating scale is recommended as a survey a 

school system that is beginning to experiment with open education. 
Baseline data can be gathered and the measure can be repeatedly used 
to chart changes in classroom practices. A teacher questionnaire, 
parallel in form to the classroom observation rating scale, may . 
used in workshops as a starting point for a dialogue about teaching. 
Both measures can be considered as initial steps in adding L 
theoretical precision and empirical understanding to the concept of 
open education. (Author) 
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Abstract 



A classroom observation rating scale, based upon a content 
analysis of the literature and conceptually verified by open 
education advocates, effectively differentiated British and 
American open classrooms from American traditional classrooms. 
The influence of socio-economic settings was also demonstrated. 
For the three comparison groups, more features of open education 
were found in higher socio-economic settings than in lower ones. 

Rater-reliability for the classroom observation measure was 
high. The classroom observation rating scale is recommended 
as a survey instrument in a school system that is beginning to 
experiment with open education. Baseline data can be gathered 
and the measure can be repeatedly used to chart changes in 
classroom practices. 

A teacher questionnaire, parallel in form to the classroom 
observation rating scale, may be used in workshops as a start- 
ing point for a dialogue about teaching. Beth measures can be 
considered' as initial steps in adding greater theoretical 
precision and empirical understanding to the concept of open 
education. 
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I. INTRODUCTION . 



"Open education," "integrated day," "Leicestershire model," and 
"activity-centered learning" are used to describe an educational 
approach which is established in Great Britain and is growing in 
tKe United States, especially among elementary school practitioners. 

I 

/ Advocates of "open education" stress an environment of manipulative 
* ' materials, choice for students, and flexibility in the use of time, 

space, materials and school requirements. At the heart of this 
movement is a philosophy of education which is reminiscent of the 

I 

American progressive era. Open education has not only been influ- 
enced by the current trend in British educational reform but also by 
a timeless amalgamation of values, such as respect for children, 

; honesty, trust, and an image of teachers who seek opportunities for 

| \ 

j personal growth. 

i • j 

• _ i 

; This investigation sought the answers to four questions: 1) Could 

j . distinctive features of open education be derived from the literature? 

t 

j 2) Were there characteristics of the open classroom which open 

t ‘ 

! , education theorists and practitioners could agree upon as essential? 

\, 

1 3 ) Could a classroom observation rating scale be constructed that would 

; distinguish open classrooms from traditional ones? Did the attributes 
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of open education more frequently appear among classrooms selected 
from middle and upper-middle socio-economic settings than in class- 
rooms in lower and working class settings? It was expected that 
middle and upper-middle class open classrooms would show higher 
scores on the classroom observation rating scale than working and 
lower class open classrooms. 

The reader of current articles and books about open education is 
perhaps left with an uneasy feeling that only exceptional teachers 
are being described or that an unobtainable picture of teaching is 
being presented. Another cause for confusion is the eclectic use 
of the term, open, for anything that is an innovation, i.e. , adop- 
tion of a new school program, individual instruction, team teaching, 
affective education, or architectural charges. To equate open 
education with many different contemporary educational practices 
may cause the knowledge, experiences, and learnings of open 
education proponents to be misunderstood or rejected by people who 
are responding to stereotypes. Its critics call it vague (Etzioni, 
A., 1971), while its adherents continue to write about it at an 



See, Kohn, Melvin (1969) for an excellent summary of different, 
child-rearinfe values according to socio-economic status. Briefly, 
he discusses two polar sets of values} one set emphasizes self- 
direction and internal standards of behavior and the other set 
stresses authority and externally imposed standards. The first set 
is characteristic of middle class groups while the latter is more 
frequently associated with lower or working class groups, according 
to Kohn's analysis. 



t 

I 

i 

( 

\ ; 

\ 

increasing rate (Silberman, C., 1970; Barth, R. and Rathbone, C., 
1971; Featherstone, J., 1971; Murrow, C. and L. , 1971). 



One diff icultyNin analyzing open education is that it is conceived of 

as an evolutionary process; that is, change is always occurring. 

Static models, or even educational models per se, prescriptions, 

given sets of definitions, or the use of behavioral objectives are 

deliberately avoided by open education advocates; Instead, emphasis 

is given to the unique child, specific events, the intuitive reactions 

of teachers and students engaged in the process of learning. The 

key work is process. Eisner's succinct analysis of expressive and 

instructional objectives is pertinent: 

Expressive objectives differ considerably from in- 
structional objectives. An expressive objective 
does not specify the behavior the student is to 
acquire after having engaged in one or more learning 
activities. An expressive objective describes an 
educational encounter: it identifies a situation in 

which children are to work, a problem with which they 
' are to cope, a task they are to engage in--but it 
does not specify what from that encounter, situation, 
problem, or task they are to learn. An expressive 
objective provides both the teacher and the student 
with an invitation to explore, defer or focus on 
issues that are of peculiar interest or import to the 
inquirer. An expressive objective is evocative 
rather than prescriptive. 2 





2 

E. Eisner. Instructional and expressive educational objectives: 
Their formulation and use in curriculum. In AERA Monogram Series 
on Curriculum Evaluation; Vol. 3, Instructional Objectives . 1 
Chicago: Rand McNally, 1969, pp. io -16 as quoted by Bussis & 

Chittenden, 1970. 

-3- 
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Eisner's distinction is useful in highlighting open educators' 

concern with expressive objectives as opposed to researchers' use 

* ’ X, 

of instructional objectives. Researchers often use only perform- 
ance measures such as intelligence and achievement scores to 
evaluate their work. In assessing open education, an important 

initial stage is determining whether open classrooms are operating 

/ 

as their proponents suggest. 



Because open education is relatively new in the United States, 

* / 

this stu<iy focused upon identifying and measuring features of the 
learning environment rather than evaluating the relative "success" 

^ of open, classrooms . If one is interested in educational outcomes of 

; : 

open classroons, the careful, longitudinal studies of achievements 
of children in British integrated day classrooms by D. E. M. 



Gardner (1950, 1965, 1966) are documents which present, important 



research findings and guidelines. As open classroom processes 
become better identified, it seems reasonable to expect ;that a 



variety of outcome measures', similar to ones used* by Gardner, will 



be employed to satisfy questions raised by parents, educators, and 
teachers about the long-term effects of open education. 



Gardner's overall findings werd* favorable for the British 
integrated day classrooms compared (to British traditional class- 
rooms, although tie traditional .classrooms were not as carefully 
selected as thd experimental, integrated day classrooms. 



II. 



METHOD 




A. Construction of Open Classroom Measure 

There are two basic ways to measure classroom environments: 

1) low inference categorical systetns and 2) high inference rating 
scales. The former are generally preferred by researchers (Simon 
and Boyer, 1967) since it is easier to systematically count specific, 

discrete units of behavior; for example, the number of open-ended 

’ / • 
f 

questions the teacher asked or £Ke number of directions given. 

Rating scales stress more global characteristics, such as enthusiasm, 
creativity, intellectual stimulation, which are high inference 
items and difficult to define operationally. When observers are 
given appropriate training, rating scales can achieve a high 
degree of inter-rater agreement, effectively discriminate among 
groups (Rosenshine, 1970), and perhaps have a meaning closer to the 
reality, of the phenomena observed than do categorical systems. 

A rating scale was used for this study because preliminary obser- 
vation instruments revealed that many materials and frequent 
movement from one activity to another within open classrooms could 
not be easily or reliably counted within a two-hour observation 

period. 
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The rating scale structure was suggested by a recent report by Bussis 
and Chittenden (1970)^ which identified ten dimensions as potentially 
valid indices of open education. For this study eight of the ten 
dimensions were isolated (Walberg and Thomas, 1971). To establish 
the content validity of these major dimensions, items indicative of each 
were identified, based upon quotations taken from the literature. The 

eight dimensions were: 

/ ' 

1. Provisioning for learning : flexibility in the 

organization of instruction, materials. 

2. Diagnosis : less attention to goals, such as 

examination scores, and more attention to the 
child's thinking process. 

3. Instruction: much individual attention rather 

than solely total class instruction, encourage- 
ment of children's initiative and choice, inter- 
disciplinary emphases. 

/ 

A. Evaluation:' individual standards or goals preferred 
to comparing the child to standardized achievement 
norms. Record-keeping often done in order to 
evaluate growth rather than correctness. 

5. Humaneness : teachers have characteristics such as 

respect for children, openness, and warmth. 

6. Seeking opportunities to promote growth : extensive 

use of community, colleagues, advisots. 

7. Assumptions : ideas about children and the process of 

learning. Many ideas are stressed such as children's 
innate curiosity, trust in children's ability to make 
decisions, and so on. 



8. • Self -percep tion of the teacher : a sensitive, adapt- 

able, continual learner who sees himself as a resource 
for helping children reach their own potentials 
rather than seeding himself as a disseminator of a. 
given body of Knowledge. 

11 



7 



%> 

A 

A 106- item questionnaire was sent to forty-one open education "experts" 
who had written articles and books about open education, were fre- 
quently quoted in the^iedU^r-were^^olved in open education 
projects. They were asked to rate each item as "very important, 
"relatively important," or "not important" for open education. 

Twenty of the experts returned the questionnaire and another ten 
sent letters giving general reactions to the questionnaire or 
explaining why they could not complete it. Extensive interviews 
were also held with several experts in order to clarify conceptions 
or to reduce confusion in the wording of items. 

After carefully reviewing the responses from the quest-iohnaire, and 
evaluating the general comipents made by the experts, a 50- item 
classroom observation rating scale was constructed. Half of the 
items were drawn from the Provisioning for learning dimension, 
and half were selected from the remaining seven dimensions. All 
the dimensions did not receive the same emphasis because of criticisms 
of many open educators and reservations of the research staff about 
the nature of certain dimensions. For example, the items written for 
Humaneness or Assumptions about children's learning were often 
considered platitudes or cliches. Further research might analyze 
the relative importance of each dimension in differentiating open 
versus traditional classrooms; but for this study, the total score 
based upon all eight -dimensions was used. 

.12 
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Classroom observers used a four-point scale for each of the 50 items. 

i ■ 

A A rating meant strong frequent evidence of the characteristic, a 
3 meant moderate occasional evidence, a 2 meant weak infrequent 
evidence and a 1 meant no evidence or anegative case of the 
characteristic. In order to avoid a response set, opposite state- 
ments about open classroom characteristics were also constructed. 

In determining the mean scores for the total observation ratings, 
all scores for statements identified with traditional classrooms 
were rotated so that a high score on all of the items would be con- 
sidered indicative of an open classroom while a total low score 
would reflect a traditional classroom. (See Appendix A for the 
scoring key and Appendix B for the classroom observation rating 

scale) 

To check the concurrent validity of the observation measure, teachers 
«ho were observed were asked to complete a questionnaire of 50 items 
parallel in meaning with the classroom observation items. An 
example oi one of the parallel items is number 17; on the classroom 
observation form it reads, "Teacher prefers that the children not 

I ‘ ' 

talk when they are supposed to be working." On the teacher 
questionnaire it is, "I prefer that children not talk when they 
are supposed to b/working." A copy of the teacher questionnaire 
is provided in Appendix 0. The same scoring procedures used for the 
classroom observation measure were used for the teacher questionnaire. 

I V 

! . v . 

13 
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B. Sample Selection 



The three comparison groups of classrooms (21 U. S. traditional, 

21 U. S. open, and 20 British open) were drawn from a wide socio- 
economic range. Children in these classrooms were between the ages 
of five and eight. U. S. traditional classrooms were matched against 
U. S. open classrooms on the basis of age level, locale, socio-economic 
status, racial and ethnic composition, and public or private school 
status. The same procedures were used for selecting classrooms in 
England. U. S. Classrooms were selected from one mid-western and two 
eastern cities; British classrooms from four major cities in England. 

To insure that there was a comparable esprit de corps for the traditional, 
or perhaps more accurately called "unaligned 1 * classrooms, school 
administrators and educational consultants recommended classrooms which 
were considered exemplary traditional rooms. First year teachers were 
not included in the study; and open classroom teachers were selected 
only i^ they had used an open classroom teaching style for at least one 

year. 

* 

Approximately half of each comparison group was drawn from a working and 
lower class population and half from a middle to upper-middle <plass 
population. For overall comparisons, the United States sample 
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represented a greater socio-economic range than the British sample 
because upper-middle class private schools were used in the two U.S. 
groups, whereas no private school classrooms were used in the British 

sample. 

In the United States, each classroom was visited by an individual 
observer three times ; in England, because of time constraints and 
economic considerations, each classroom was visited only twice. At 
least two different observers visited each classroom. In the United 
States, classrooms were visited during the latter part of March and 
the entire month of April, 1971. The English open classrooms were 
visited at the end of April and during the first week of May, 1971. 

The British observations were done immediately at the start of their 
second school semester. This is important to note because items 
designed to assess student initiative and an abundance of student- 
made products may not have been fully assessed since new student 
projects were just getting started. 

I C. Training Classroom Observers v 

There were differences in the selection, training, and composition 
i of classroom observers for the United States and England, The. United 

l 

i ■< 

\ ^ \ 

| States'" training sessions were more rigorous than those conducted in 

1 

England because the research staff had better training facilities, con 
venient access to schools, and more contacts for attracting temporary 

- 10 - 
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classroom observers. 

In the United States, a research staff composed of three women 
conducted an intensive three-day training and screening program 
for thirteen women who applied for temporary classroom observers 
positions. Two part-time researchers, who were familiar with the 
study, were trained along with the 13 applicants. All of the indiv- 
iduals trained were white, college-educated women, between the ages 
of 22 and 45. 

Separate reliability analyses were done for the research staff and for 
those trained and hired temporarily to do classroom observations. 
Because the research staff had been involved in designing the study 
and had "inside" information about the chosen schools and teachers-- 
the research staffs* scores are reported as "inside observers, the 
others as "outside observers." 



Care was taken during the training period not to reveal the purpose 
of the study. Observers were told that the research staff were 
■4 withholding information in order not to bias their perceptions and 

to avoid having them discuss details of the study with the participating 
teachers. If questioned by the teachers, they were told to say that 
they were participating in a study of contemporary classroom practices 
of excellent teachers. . The words, "experimental," "control, " 
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"traditional," and "open," were not used by the research staff during 

the training period or in the actual study. 

/■ 

Films of open and traditional classrooms were shown the first day 
of the training session.- The classroom rating scales were discussed 
when observers rated each film. The next two days, trainees in pairs 

visited an open classroom and a traditional classroom and used the 

£ 

classroom observation measure under actual classroom conditions. 
Observer ratings for the two films and two field visits were collected 
for purposes of screening applicants ■ and for gathering preliminary 
observer rater-reliability data. 

Eleven of the thirteen applicants and five members of the research 

i 

staff were used in the study. Two. applicants were eliminated from 
the study because of unreliable rating scores. About one-fourth of 
the observations in the United States were made by "inside" observers, 
the remaining three-quarters by the "outside" observers. 

In England, one research staff member trained the two "outside" 
observers during a one-day training session by carefully reviewing 
the classroom observation rating scale. Neither films nor field 
visits were used. Two "outside" observers, both women, were re- 
commended by British officials and educators as being perceptive, 
conscientious individuals, familiar with the open education concepts, 

I 

- 12 - 
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Because the British "outside" observers were chosen on the basis of 
their familiarity with the open education approach, the purpose of the 
study was not disguised for them as it was for U. S. "outside" 
observers. A total of two "inside" observers and two "outside" 
observers visited all of the English classrooms. 

/ 

I. 




\ 0 
\ erjc 
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III. FINDINGS 



A. Description of Sample 

Observers* records were used to gather background information 
about class size and the number of adults present in the class- 
room. The teacher questionnaire was ,used to provide teacher's 
age, educational background, and to determine whether the children 

were from the same or mixed age groups. 

/ 

\ 

i 

/ ' 

The class size differed significantly for the three comparison 

groups. (F = 51.388, df 2.59, £<.001) There were more children 
in the British open classrooms than in the other two groups; i.e., 
in the U.S. traditional classrooms, the mean number of students 

present was 19.8 with a standard deviation (s.d .) of 3.64; the 

*\ 1 

U.S. open was 20.8 with an s.d . of 3.55; and the British open 

» 

was 31.8 with an s.d. of 4.35. Class size was probably constant 
for all groups despite the fact that some children were hot 
counted because they were either absent or out of the room when 
the observers recorded the number of children present. 
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The U.S. open classroom had significantly more adults present 
than either the U.S. traditional or British open classrooms, 
i.e., more teacher aides, student teachers, parents, and uniden- 
tified visitors. (F = 16.893, df 2.59, £ < .001) x Including 
teachers, in the U.S. traditional classrooms there were 1.71 
adults (s.d . 1.56); U.S. open, 2.49 adults (s^. 0.94); and 
British open, 1.35 adults (s.d . 0.77). The ages of teachers 
in the three groups were not significantly different. The 
U.S. traditional teachers' mean age was 34.8; the U.S. open 
teachers' age was 35.2, and the British open teachers' age 
was 32.6. 

The British teachers held significantly more normal school or 
teacher college degrees than the U.S. teachers, traditional and 
open, who held more master's degrees. (X , df 4, £<.001) 
Thirty-eight percent (8 of 21) U.S. open classrooms and thirty- 
five percent (7 of 20) British open classrooms contained children 
of mixed ages. There were no mixed age groups in the U.S. 



traditional classrooms. 
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B. Item Analysis 

Point bi-serial correlations for individual scores with two 
variables, open or traditional labels, showed that 43 items 
differentiated the two open classroom groups from the traditional 

group on the classroom observation rating scale. 

\ ' • ' 

• .. \ 

' \ | 

The items which correlated significantly with the open class- 
room variable show that open classrooms demonstrated more variety 
in the use of materials and activities and more flexibility in 
grouping and scheduling procedures. The children^ jked and moved 
about the room more frequently and seemed to be more deeply 
involved in what they were doing than children in traditional 

n * 

classrooms. • 

The traditional classes, by contrast, showed greater concern with 
academic achievement, used tests for evaluation purposes , and 
standardized curriculum.' The children in the traditional classrooms 
seemed to expect the teacher to correct much of their work and 
the teachers preferred the children not to talk as they worked 
or to move about the room. See Appendix D for the complete list 

of items and the point bi-serial correlations. 

I . 

I 

\ 

O 

• < 

4 . •' 
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It is useful to note that of the seven non-discriminating Items 
two (No. 12 and No. 15) were drawn from the Provisioning category; 
two (No. 35 and No. 47) were from the Evaluation category; one 
from Seeking . (No. 44 ) ; one from Humaneness (No. 40); one from 
Instruction (No. 22). The seven items as a whole appear to be 
of a more highly inferential nature than many of the more objective 
statements on the measure. Only two of the 25 Provisioning items 
were^found^to^ be non- discriminating and they appear to be more 
subjective than others in the Provisioning category. 

% t I 

* s * 

One item, No. 40, '*The teacher is in charge," merits special 
attention. This item was inserted into the rating scale following 
criticism of the preliminary questionnaire. Many open educators 

felt that items about open classroom teachers suggested a laissez- 

* 

faire and irresponsible quality. The mean observation scores on 
No. 40 for all three groups substantiated open educators' avowal 
that the open classroom teachers are responsible and do not 
abdicate authority (U.S. traditional, 3.63; U.S. open, -3. 55; and 
British, 3.70). 

Seven items on the teacher questionnaire, Nos. 22, 35, 37, 38, 40, 

44, and 47, did not correlate with the variables open or traditional 

. / < 
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Five of these items. Nos. 22, 35, 40, 44, and 47, were also non- 
discriminating items for the observation measure. See Appendix E 

for item means and point bi-serial correlations for the teacher 

( * 
questionnaire. 

The teacher questionnaire total scores were correlated with the 

total scores for classroom observations. For the full sample of 

62 classrooms there was a highly significant correlation of .782. 

* 

For each classroom group the correlations between the total quest- 

• * 

,‘ionnaire scores and the total observers' scores were: U.S. trad- 

itional, .747; U.S. open, .369; and British open, .177. The 
correlations for each ^group were of a smaller magnitude than for 
the total sample probably because each classroom group was more 
homogeneous thsin the combined total sample. 



The reliability of the teacher questionnaire, using Cronbach's 
alpha method was: for the total sample, .916 with a standard 

error of measurement of 5.26; for the U.S. traditional group, .848 
with a standard error of measurement of 5.26; and for the British 
open group, .836 with a standard error of measurement of 4.86. 
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C. Rater-Reliability 

Classroom rater-reliability for the rating scale was determined 
by Cronbach's alpha method. Table 1 shows that the reliability 
scores for "inside," "outside," and combined groups of observers 
were quite high. 

TABLE 1 



3 
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Condition 


Inside 

Observers 


Outside 

Observers 


Total 


U.S. Open Group 


.721 


.923 


.930 


U.S. Traditional Group 


.932 


.807 


.864 


British Open Group 


.848 


.902 


.876 



Table 2 shows that during the training sessions the "inside" 

observers had consistently higher reliability scores than the 

/ 

"outside" observers although both groups had unreliable scores 
for the open classroom field testing. The very high reliability 
scores for both groups during the actual study shows a marked 
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improvement . 



TABLE 2 



Condition 



Inside 

Observers 



Outside 

Observers 



Tot^al 

1 ' 



I 



U.S. Traditional, film 


.913 


.610 


* 74 f V 


U.S. Open, film 


.882 


.509 


.659 - 


U.S. Traditional, 
field practice 


.911 


.786 


.802 


U.S. Open, field practice 


.032 


-.075 


-.085 



D. Analysis of Variance 

A 3x2 analysis of variance (groups by socio-economic status) showed 
that there was a highly significant difference among the three 
comparison groups as well as a significant socio-economic effect. 
See Table 3. 

TABLE 3 




Source 



df 



MS 



0 
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Socio-Economic Status 


(SES) 1 


1331.46 


5.94 


.0180 


Group 


/ 2 

• / 


14034.80 


62.63 


.0001 


SES x Group 


. / 


275.75 


1.23 


.3000 


Error 


56 


224.13 
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According to the results of the open classroom measure, both 
the U.S. and British open classrooms were significantly different 
from the U.S. traditional classrooms (£<.01), while no significant 
differences were found between the British open and the U.S. open v 
classrooms. 

There was a great amount of variability for all three groups as 
seen in Figure 1. The observation mean scores could theoretically 
range from zero to 200, with 200 being the "ideal" open score. 

The mean score for the U.S. traditional group was 117.46 with a 
standard deviation of 19.59; the U.S. open scores was 163.17, 14.08; 
and the British open was 160.80, 13.07. 

Many of the classrooms in all three groups had similar scores. 

For example, six U.S. traditional classrooms, four U.S. open, and 
seven British open classrooms had scores between 130 and 154. 
However, for 68 percent of each group (one standard deviation) the 
classroom observation scores did not overlap between the two open 
classroom groups and the U.S. traditional group. 

Figure 2 illustrates the analysis of variance finding that there 
was a significant socio-economic effect, i.e., more open classroom 
features would be found in middle and above classroom groups than 
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would be found in comparable classrooms from a working and 
lower class socio-economic . setting* Within all three groups 
the middle and above socio-economic groups had higher mean 



classroom observation scores. 
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Figure 1 * - Average of observer ratings for each 
U.S. traditional, U.S. open, and British open classroom 
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Figure 2. - Mean observation scores for U.S. traditional, 
U.S. open, and British open classrooms by socio-economic status 



IV. DISCUSSION 



\ 



The classroom observation rating scale effectively demonstrated 

V. 

that two different pedagogical styles could be theoretically 
identified on the basis of the literature and empirically verified 
in the field. This general finding tends to refute the common 
assertion that open education is vague and imprecise. 



The rating scale showed that expectations for children, the 
physical arrangement of classrooms, the role of the teacher, 
the use of curriculum materials and tests, the direction of act- 
ivities, the use of time and priorities for children were fund- 

' . / 

ament ally quite different for the open and traditional groups. 



The traditional teachers were . much more in/ control of the learning 
environment with regard to- organizing the child* s use of time, 
materials, space, and the curriculum to be studied. They- expected 
children not to talk while working, nor to move about without asking 
permission. The physical environment was uniformly arranged so 
that children could conveniently see the blackboard or the teacher 

from their desks. The teacher stressed keeping all children within 

\ ' ' 
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his sight so that he could make sure they were doing. what they 
were supposed to do. In general, the children were supposed to use 
standardized curriculum materials, and the teacher gave academic 
achievement a top priority. Testing was used by the teachers 
for grouping the children and for grading them in comparison with 

their peers. 

The open classroom teachers, by contrast, allowed the children 
more freedom in the use of time, choice of activities, and ways of 

r 

working. The children worked individually and in smali groups at 
various activities, which often involved the use of manipulative 
materials. The children used '•books” written by their classmates 
as part of their reading and reference materials, and often children 

spontaneously looked at and discussed each others' work. The 

\ 

teacher concentrated his time with the children by providing 
intensive diagnostic help rather than giving whole group instruction. 
Children were encouraged to use other areas of the building and 
school yard during school time. The children seemed deeply involved 

in what they were doing. \ 

• / 

/ 

/ 

/ 

More individualized record-keeping was expected for the open 
classrooms because the literature stressed the value of this form 
of evaluation as an alternative to traditional testing. However, 

30 
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keeping notes and writing individual histories of each child's 
intellectual, emotional, and physcial development was found 
only occasionally in the open as well as traditional classrooms. 

Another major finding concerns the influence of the socio-economic 

setting upon classroom characteristics. More features of open 

\ 

education were found in upper socio-economic classrooms for all 
three comparison groups; and it should be noted that the lower 
socio-economic open groups in England and in the United States 
had more features of open education than did the higher socio- 
economic traditional groups. 

It is important t& recognize several limitations to this study 
and to distinguish where the use of the 50-item rating scale is 
appropriate, and where it is not appropriate. 

' l 

First, the rating scale was not designed to assess all the diff- 
erences among the British open, U.S. open and U.S. traditional 
classrooms. For example, it appears that there are more students 
in British primary classrooms than in either of the U.S. classroom 
groups, but far more adults were present in the U.S. open classrooms 
than in the other two groups. Quite possibly the U.S. open class- 
rooms are attracting more student teachers, aides, visitors, and 
parents because open education is a novelty in the United States. 



Second, some of the items describing the open classroom also 

* 

describe what traditional educators, would value as good teaching 
practices. This is particularly true of the more inferential 
items, such as item 25 , "The teacher promotes a purposeful 
atmosphere ...;' 1 item 30 , "The einotional climate is warm and 
accepting;" and item 50 , "The children are deeply involved in 
what they are doing." . ' 

Third, the socio-economic setting is just one of many variables' 
influencing educational practice. The finding that the higher 
the socio-economic setting of a school, the greater is the like- 
lihood that more features of open education will be present is 
subject to qualification. In this study, three of the top four 
classrooms in the U.S. open sample were from black, inner-city 
schools. According to observers and research staff members, 
these top-rated classrooms probably illustrate the effects of 
a committed staff, approval of school administrators , as well 
as the support of informed, sensitive advisors. 

Fourth, many methodological improvements could have been made, 
such as more classroom visits, an equal number of visits for both 
U.S. and British classrooms, similar training procedures for U.S. 
and British observers, and greater care in the selection of 
British classrooms and in the timing of the visits in England. 



I 



Finally, we need to ask why both groups of observers demonstrated 

\ . .. 

high reliability scores during the actual study when they had 
scored so. much lower for the open classroom field training* 

There may be three reasons for this discrepancy: (1) two observers 
who had very unreliable scores during the training session were not 
used in the actual study; (2) several observers who had moderately 
unreliable training session scores were given additional tutoring 
in the use of the measures; and (3) as all the observers visited 

v , • 

a wide variety of classrooms, the definitions of each item and how 
it should be rated became clearer and ratings more consistent. 

Great care, therefore, must be taken in both the selection and 
training of observers. . 

When arjd where can the classroom observation measute be used? The 
rating scale is probably best used as a survey instrument in a 
school system that is beginning to experiment with open classroom 
techniques. It is less reliable as a diagnostic measure for indiv- 
idual classrooms. An experienced observer can rate the 50 items 
in only a few minutes, making it possible for baseline data to be. 
gathered and changes charted over a period of time in a large 
school system. j 

' V 

It must be remembered, however, that the measure was designed for 
primary grades and' therefore, may not be appropriate with other 
normative groups. Furthermore, the items cannot be expected 
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to produce diagnostic insights one could expect of an exper- 
ienced advisor who observes a single classroom over a long 
period of time. Any kind of fixed format, such as that used in 
the rating scale, does not allow for the individual counseling 
most teachers would like. Something tailor-made to teachers 
p'roblems , goals, and teaching styles, as well as organizational 

A 

constraints, demands the kind of imaginative, professional insights 

of a sensitive observer who tries to understand the gestalt of 

the classroom. Certainly the classroom rating scale could be 
* / 

used by such an observer as a check list f<pr points to consider 
in talking with teachers. 



The rating scale, while useful in discriminating open from 
traditional classrooms, does . not effectively discriminate key 

features of outstanding open classrooms, i.e., a "low ceiling 

. . ! 

effect" is probably operating because several open classrooms had 
extremely high scores on the rating scale. 



Some researchers or school personnel may not have the time or 
money to train observers to use the classroom observation measure. 

An obvious question is therefore, "Can I have the teacher use the 
teacher questionnaire to rate himself?" The answer is "yes" and "no 
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Yes, the teacher questionnaire can be used for self-evaluation. 
The teacher may find it helpful to look at a variety of specific 
classroom characteristics or a number of teachers at a workshop 
or given school may use it as the basis for a dialogue about their 
teaching. However, if one wants an objective evaluation, the 
answer is "no/." If teachers insist upon using the questionnaire 
as an objective pedagogical test,' they are not using it properly. 
Questionnaires of any kind can easily be "halted." Furthermore, 
there is always an error factor associated with testing. 



One can use the standard error of measurement for a test to 

estimate the parameters around which the "true" scares should 

\ 

lie sixty-eight percent of the time. For example, in this study 
for the total sample, the standard error of measurement of the 
questionnaire was 5.96. If a teacher received a measure score 
of 150 on the questionnaire, it is likely that his true score 
would probably be plus or minus 5.96 points, or usually between 
144.04 and 155.96. However, probability does not guarantee that 
his true score will fall within this range. The teacher question 

i j 

naire scores differed by as much as 25 points from the observers’ 
averaged ratings. An evaluation study that compares teacher 
questionnaire scores with observers’ scores could give teachers 

feedback about discrepancies in how they saw their classrooms as 

. / 

opposed to how observers viewed them. 
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To summarize, the classroom observation rating scale may be 
valuable for gathering baseline data in school systems which 
are trying to adopt and measure key features of open education; 
the teacher questionnaire may be useful for teachers in a work- 
shop as initial points for dialogue; or both may be used as 
measures for other researchers to incorporate into their own 

I 

work about contemporary teaching. 

For future research it would be useful to see how character- 
istics of the classrooms, as assessed by the rating scale, relate 
to students* self-perceptions and achievement measures. School 
administrators, teachers, and researchers are encouraged to use 
the classroom observation rating scale and teacher questionnaire 
without contacting Education Development Center (EDC) for permission, 
although they are asked to send to EDC copies of reports based 
upon the measures in order to keep the open education literature 

up-to-date. 

\ • ' 
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SCORING KEY 

WITH WEIGHTED ITEM SCORES 

FOR CLASSROOM OBSERVATION RATING SCALE AND TEACHER QUESTIONNAIRE 



y 



ITEM 



POSITION 
1 2 3 
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1. Texts and materials are supplied in class sets 
so that all children may have their own. 



2. Each child has a space for his personal storage 
and the major part of the classroom is organized 
for common use. 



3. Materials are kept out of the way until they 
are distributed or used under the teacher's 
direction. 



4 . Many different activities go on simultaneously. 



3. Children are expected to do their own work 
Without getting help from other children. 



5. Manipulative materials are supplied in great 
diversity and range, with little replication. 



7. Day is divided into large blocks of time 
Within which children,. with the teacher's help, 
(determine their own routine. 



jj. Children work individually and in small groups 
'at various activities. 



. Books are supplied in diversity and profusion 
including reference , children's literature). 
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lip. Children are not supposed to move about the 
jrdom without asking permission. 

} ' . 

11. Desks are arranged so that every child can see 
jthe blackboard or teacher from his desk. 

I 

I 

12. The environment includes materials developed 
by the teacher. 
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13. Common environmental materials are provided. 

i \ 

i 

i 

14. Children may voluntarily make use of other 

. lareas of the building and school yard as part of their 
'school' time. 

; j 

j 

| ’ . 

L5. The program includes use of the neighborhood. 

i 

i.6. Children use "books" written by their class- 
mates as part of their reading and reference 
jiaterials, 

j 

s 

i 

i 

.7. Teacher prefers that children not talk when 
hey are supposed to be working. 

8. Children voluntarily group and regroup 
hemselves. 

I 

19. The environment includes materials developed 
|r supplied by the children. 

i 

! 

|o. Teacher plans and schedules the children's 
ctivities through the day. 

1. Teacher makes sure children use materials only 
3 instructed. 
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22. Teacher groups children for lessons directed 
at specific needs. 

I 

i 

j 

• 23. Children work directly with manipulative mater- 
| ials. 

( 

f 

! 24. Materials are readily accessible to children. 

i 25. Teacher promotes a purposeful atmosphere by 
j expecting and enabling children to use time 
| productively and to value their work and learning. 

! i 

! 26. Teacher uses test results to group children 
; for reading and/or math. 
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| 27. Children expect the teacher to correct all 
their work. 



12 3 4 



! 28. Teacher bases her instruction on each 
! individual child and his interaction with 
! materials and equipment. 



j 29. Teacher gives children tests to find out what 
| they know. 

i 

1 

! 30. The emotional climate is warm and accepting. 

! 

i 

i 

j 31. The work children do is divided into subject 
I matter areas. 

32. The teacher's lessons and assignments are 
given to the class as a whole. 

j 33. To obtain diagnostic information, the teacher 
j closely observes the specific work or concern of a 
S child and asks immediate, experience-based questions. 
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34. Teacher bases her instruction on curriculum 
i guides or text books for the grade level she 

; teaches. 

i . \ 

I 

1 

35. Teacher keeps notes and writes individual 
histories of each child's intellectual, emotional, 
physical development. 

j j ■ . / . 

; 36. Teacher has children |for a period 7 of just one 

i year. . ' 

[ 37. The class operates within cle^r guidelines 
! made explicit. 



/ 



/ 



I 38. Teacher takes care^of dealing with conflicts 
I and disruptive behavior without involving the group, 



39. Children's activities, products, and ideas are 
reflected abundantly about the classroom. 



40. The teacher is in charge. 



: 41. Before suggesting any extension or redirection 
i of activity, teacher gives diagnostic attention to 
i the particular child and his particular activity. 
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42. The children spontaneously look at and discuss 
each other's work. 



j 43. Teacher uses tests to evaluate children and rate 
: them in comparison to their peers. 



j 44. Teacher uses the assistance of someone in a 
supportive, advisory capacity. 

j 45 . Teacher tries to keep all children within her 
| sight so that she can make sure they are doing what 
I they are supposed to do. 
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46. Teacher has helpful colleagues with whom she 
discusses teaching. 

47. Teacher keeps a collection of each child's work 
for use in evaluating, his development. 



48. Teacher views evaluation as information to 
guide her instruction and provisioning for the 
classroom. 



49. Academic adhievement is the teacher's top 
priority for the children. 



50. Children are deeply involved in what they are 
doing . 
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School 

Classroom 

Teacher 
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QUESTIONNAIRE 



Instructions: For each of the following statements, circle the number 

which most closely expresses your estimate of the extent to w 
statement is true of your own classroom. If the state e y 

not the case, circle ”1”; if it is very minimally true, choose 2 If 
the statement generally describes your classroom, choose 3 , it it is 
absolutely true choose " u ." 



strongly 

disagree disagree agree 



1. Texts and materials are supplied in class sets 
so that all children may have their own. 



2. Each child has a space for his personal storage 
and the major part of the classroom is organized 
for common use * 



3. Materials are kept out of the way until they 
are distributed or used under ny direction. 



4. Many different activities go on simultaneously. 



5. Children are expected to do their own work 
without getting help' from other children. 



6. Manipulative materials are supplied in great 
diversity and range, with little replication. 



7. The day is divided into large blocks of time 

within which children, with my help, determine 1 2 3 

their own routine. 



8. Children work individually and in small groups 
at various activities. 
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strongly 

disagree 



9. Books arc supplied in diversity and profusion 
(including reference books, children's literature). 

10. Children are not supposed to move about the room 
without asking permission. 

11. Desks are arranged so that every child can 
see the blackboard or teacher from his desk. 

12. The environment includes materials I have 
developed. 

13. Common environmental materials are provided, 

14. Children may voluntarily use other areas of 
the building and schoolyard as part of their school 
time . 

15. Our program includes use of the neighborhood. 

16. Children use ’’books” written by their class- 
mates as part of their reading and reference materials. 

17. I prefer that children not talk when they are 
supposed to be working. 

18. Children voluntarily group and regroup them- 
selves . 

1?. The environment includes materials developed 
or supplied by the children. 

20. I plan and schedule the children's activities 
through the day. 

21. I make sure children use materials only as 
instructed. 
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strongly 

disagree disagree agree 



22. I group children for lessons directed at 
specific needs. 

23. Children work directly with manipulative 
* materials . 

2u . Materials are readily accessible to children. 
« 

25. I promote a purposeful atmosphere by expect- 
ing and enabling children to use time productively 
and to value their work and learning. 



26. I use test results to group children in reading 
and/or math. 

27. Children expect me to correct all their work. 



28. I base my instruction on each individual 
child and his interaction with materials and equip- 
ment . 



29. I give children tests to find out what they 
know. 

30. The emotional climate is warm and accepting. 



31. The work 
matter areas. 



children do 



is divided into subject 



1 2 



3 



32. My lessons and assignments are given to the 
class as a whole. 



33. To obtain diagnostic information, I observe 
the specific work or concern of a child closely 
and ask immediate, exp€rience-based questions. 
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strongly strongly 

disagree disagree agree agree 



34. I base my instruction on curriculum guides or 
the text books for the grade level I teach. 



35. I keep notes and write individual histories of 
each child's intellectual, emotiona] , and physical 
development . 



36. I have children for just one year. 



37. The class operates within clear guidelines, 
made explicit. 

38. I take care of dealing with cor.f . icts and 
disruptive behavior without involving the group. 

39. Children's activities, products and ideas are 
reflected abundantly about the cla v-.i oom. 



40. I am in charge, 



41. Before suggesting any extension or redirection 
of activity, I give diagnostic attention to the 
particular child and his particular activity. 



42. The children spontaneously look at and discuss 
each other's work. 



43. I use tests to evaluate children and rate them 
in comparison to their peers. 

44 . I use the assistance of someone in a supportive 
advisory capacity. 

45. I try to keep all children within my sight so 
that I can be sure they are doing what they are sup- 
posed to do. 
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strongly 

disagree 



disagree agree 



strongly 

agree 



46. I have helpful colleagues with whom I 1 2 3 4 

discuss teaching ideas. 

47. I keep a collection of each child's work for 
use in evaluating his development. 

48. Evaluation provides information to guide 
my* instruction and provisioning for the 
classroom. 

49. Academic achievement is my top priority for 
the children. 

50. Children are deeply involved in what they ^ 2 3 4 

are doing througn the day. 
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Teacher's name 

School 

Location 

Present position: permanent 

provisional 

temporary 

Age: 20-25 41-50 

26-30 51-60 

31-40 over 60 



Education (check all applicable): Normal school degree 

Bachelor's degree 

Master's degree 

Other (specify) 

Address: in locality of school 

elsewhere 



Your classroom: 

Grade level (check one) 

Kindergarten Ungraded 1-3 - Ungraded K 6 

1st grade Ungraded 1 & 2 Ungraded k-2 

2nd grade Ungraded 2 & 3 Ungraded k-3 

Ability range: streamed/ability grouped 

mixed ability grouping 



Number of children 



Racial composition: 
(give approximate %) 



wni ce 

nonwhite 
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APPENDIX D 



ITEM MEANS AND POINT BI-SERIAL CORRELATIONS 
CLASSROOM OBSERVATION MEASURE 
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